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(54) Multimedia conferencing system over ISDN network 


(57) Real-time multimedia services are transmitted 
over a hybrid network including a nonguaranteed quality 
of service packet switched local area network and a cir- 
cuit switched ISDN wide area network having a central- 
ized multimedia bridge located within the wide area 
network. The local area networks and multimedia 
bridge are interconnected via ISDN routers. An algo- 
rithm executed by the multimedia bridge receives sig- 


nals from the packet switched network and detects the 
absence of properties needed for real-time audio visual 
services. The data signals are processed to compen- 
sate for the absence of the propwiies and then are 
transmitted over the wide area network to enable real- 
time audio visual services. 
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Description 
Technical Held 

This invention relates to multimedia conferencing 5 
services and more particularly relates to such services 
provided over multiple nor^uaranteed quality of service 
local area networks in geographically dispersed loca- 
tions interconnected by an ISDN wide area network in 
" wrfScfi bridging is performed in the ISDN network. w 

Background of the Invention 

Real-time multimedia services have been provided 
in the past by connecting signals ov& circuit switched is 
networks having guaranteed quality of service. 
Although this approach results in reasonable quality, it is 
cost prohibitive for many potential users. Most offices 
desiring multimedia services, such as video conferenc- 
ing, are sensed by a packet switched local area network 20 
(LAN) provkling nonguaranteed quality of service. Such 
networks are inherently unreliable for multimedia. The 
multimedia packet streams coming from such networks 
do not have the proper sequencing. The arrival time 
between the packets rmy have significant variations 25 
known as delay jitter, and some packets may be lost 
Moreover, the recovered bit streams of audio and video 
do not maintain required temporal relationships to 
insure both intramecOa and intermedia synchronization 
so that lip-synchronization is maintained. Since the cost 30 
of installing a guaranteed quality of service network in 
such offices is prohbitive. the adoption of muttimedia 
servfees in most offices has been severely restricted. 

One attenpt to provide video and audo services is 
described in US. Patent No. 4.866.704 (Bergman, filed 35 
March 16, 1988). Bergman describes the conversion of 
data to T1 frames and then an additional conversion to 
padcets for distribution on a wide band f 3)6r optic ring. 
This arrangement is contrary to the existing local area 
network structure found in most offices, and therefore 40 
would required the installation of new networks at pro- 
hibitive expense. 

Another approach to transmitting data from one 
location having a local area network to a second loca- 
tion having a local area network via a T1 network is 45 
described in US. Patent No. 5.384,835 (Wheeler et al.. 
filed January 12. 1993). Wheeler et al. teach such a net- 
work arrangement for distributing text data and still 
graphic data. However, they do not describe any 
method or apparatus for providing real time video and so 
audio over the local area and T1 networks. 

Summary of the Invention 

My invention enables real-time multimedia confer- ss 
encing service to be provided to dispersed locations 
interconneted by a wide area network from multimedia 
signals originating at the locations. The multimedia sig- 


nals are audio, video or data signals, or any combina- 
tion of such signals. The locations employ 
nonguaranteed quality of service packet switched local 
area networks which degrade one or more properties 
required by the muttimedia signals. In such an environ- 
ment, multimedia conferencing signals may be provided 
by receiving the multimedia signals from the local area 
networks at tiie wide area network, bridging the multi- 
media signals in tiie wide area network and transmitting 
tiie bridged multimedia signals over tiie wide area net- 
work to the various locations. By using this metiiod. tiie 
bandwidth required to transmit the multimedia signals 
between the dispersed locations is reduced 

According to anottier aspect of my invention, the 
absence of ttie required properties is detected in tiie 
wide area network. The nrtultimedia signals tiien are 
processed to compensate for the absence of the prop- 
erties. The processed multimedia signals are transmit- 
ted to the dispersed locations over Uie wide area 
netwcK-k in order to improve tiie multimedia services. 

My invention also may be enf^nced by detecting 
tiie absence of the properties in the multimedia signals 
at tiie dispersed locations and processing the multime- 
dia signals to compensate for tfie absent properties. 

Brief Description of the Drawings 

In the drawing. 

Figure 1 illustrates a preferred form of end-to-end 
network configurations for multipoint muttimedia 
conferencing sen^ices tiirough interconnection of 
nonguaranteed quality of service (NGQOS) local 
area networks (LANs) to an integrated services dig- 
ital network (ISDN) via ISDN routers in accordance 
with nrty invention; 

Rgure 2 d^icts a preferred form of high-level end- 
to-end protocol architecture for tiie multipoint multi- 
media conferencing services through interconnec- 
tion of the nonguaranteed quality of service 
(NGQOS) local area networks (LANs) to tiie inte- 
grated services digital network (ISDN) via ISDN 
routers in accordance wttti my invention; 
Rgure 3 shows a prefen^ed torm of nonguaranteed 
quality of service (NGQOS) local area netwaks 
(LANs) based multimedia workstationy^ersonal 
conputer (MMWS/PC) protocol architecture in 
accordance with my invention; 
Rgure 4 shows a preferred form of switched non- 
guaranteed quality of servrce (NGQOS) local area 
network (LAN) hub based multipoint multimedia 
bridging (MMB) protocol architecture in accordance 
with my invention; and 

Rgures 5 through 8 show flow charts of a preferred 
form of algorithm for intra and inter-media synchro- 
nization to inrprovement performance of the real- 
time audio and video traffic in accordance with my 
invention. 
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Detailed Description 

Figure 1 illustrates multimedia work stations or per- 
sonal computers 1-1, 1-2, 1-3. 1-4 and 1-5 that are 
equipped with multimedia conferencing application pro- s 
grams based on the H.323 standard of the International 
Telecommunications Union (ITU). Ck>mputers 1-1 and 
1 -2 are connected to a shared nonguaranteed quality of 
sennce local area network 2-1 at a cistomer premises 
6r1l^?5h"Ll . Similarly, computers 1 -4 and 1 -5 are con- to 
nected to a shared nonguaranteed quality of service 
local area network 2-2 located at another customer 
premises or location L2 which may be displaced from 
location LI by many miles. Networks 2-1 and 2-2 are 
equipped with a gateke^er function that controls the is 
total loading on the network as descrbed in ITU Rec. 
H.323. A conventional ISDN router 3-1 is located at 
location LI for connecting signals from network 2-1 to a 
conventional circuit switched narrow band ISDN wide 
area network (WAN) 5 over a link 4-1 . Signals are proc- 20 
essed in time division multiplexing fashion over the 
ISDN links 4-1 through 4-4 before being transmitted 
over network 5. Similarly, a conventional router 3-4 at 
location L2 is connected to network 5 over a link 4-4. 
ISDN routers 3-1 and 3-4 preferably have a gatekeeper 2s 
functionality. 

Still refenring to Rgure 1, a single multimedia work- 
station or personal computer 1-3 is connected to a 
switched local area network hub S-2 at another cus- 
toms premises or location L3 which may be displaced 30 
from locations LI and L2 by many miles. Such an 
anangement has superior performance when com- 
pared with networks 2-1 and 2-3. especially for nrtultime- 
dia traffic. 

Networks 2-1 and 2-2. as well as hub 5-2, can be an 35 
Ethernet (IEEE 802.3), token ring (IEEE 802.5). fast 
Ethernet (IEEE 802.10), or FDDI (opiating in the non- 
guarartteed quality of service mode). 

Still referring to Rgure 1, a nrultimedia brUge 6 is 
connected to a defeated high speed nonguaranteed 40 
quality of service local area network hub 5-1 tiiat pro- 
vides protocol transparency with networks 2-1 , 2-2 and 
hub 5-2. Hub 5-1 is connected to nruKimedia bridge 6 to 
provMe a guarantee of performance parameters, such 
as delay jitter, packet toss, en-or rate and delay. In a 45 
multipoint telephone call, a bad source may effect the 
entire conference. Bridge 6 detects the bad source and 
communicates with end conrputers 1-1. 1-2, 1-3, 1-4 
and 1-5. Bridge 6 also takes specific actions agreed 
upon at ttie time of call setup. The conrmnjnication mes- so 
sages between the computers on customer premises 
LI - L3 via ISDN wide area network 5 insures quality of 
service for nruiltimedia conferencing. 

ISDN routers 3-1 . 3-2. 3-3 and 3-4 are connected to 
local area network 2-1 . hubs 5-1 and 5-2 and local area 55 
network 2-2. respectively. The multimedia conferencing 
applications running on computers 1-1 tfirough 1-5 are 
made in accordance with the ITU Rec. H.323 standard. 


ISDN routers 3-1 through 3-4 are connected to network 
5 via ISDN links 4-1 through 4-4. respectively 

Multimedia bridge 6 is equipped witii a multipoint 
control unit (MCU) functionality in accordance witfi tfie 
ITU Rec. H.323 standard, and has a general conference 
control and multipoint communication service function 
operating in accordance witfi the ITU Rec. T120 stand- 
ard, as well as a multipoint controller and multiproces- 
sor function operating in accordance with the ITU Rec 
H.323 standard. Bridge 6 also has a gatekeeper func- 
tion, such as address translation, admission control, 
bandwidth control, call cont'd signaling, call authoriza- 
tion, bandwidth management, and call management as 
specified in the ITU Rec H.323 standard. The comnumi- 
cation between computers 1-1 ttirough 1-5 and tiie 
bridge 6 takes place in a master-slave relationship as 
described in ITU Rec H.323. Bridge 6 plays the role of 
the master in controlling participating computers 1-1 
through 1-5. 

Bridge 6 depacketizes all encapsulated pad^ of 
tiie multimedia bit sto'eams of signals that originate from 
computers 1-1 ttirough 1-5 and provides bridging for 
audio, video, and/or data. Thus, in ttiis specification and 
dain^. multimedia signals means audio, video or data 
signals, or any combination of such signals. A point-to- 
point call initially can be processed by bridge 6 if the end 
stations (i.e.. conputers 1-1 ttirough 1-5) want certain 
services, such as directory services, admission control, 
or address translation at ttie tinrie of call setip. while ttie 
actual bearer channel is estat^ished via an optimal 
patti. 

After the packets received from conputers 1-1 
ttirough 1 -5 have been d^cketized. brkige 6 executes 
an algorithm (Rgures 5-8) which serializes ttie packets, 
produces dummy packets for lost packets if appropriate, 
compensates for delay jitter where necessary and 
insures real-time audio and vkieo synchronization as 
long as ttie end stations operate within certain perform- 
ance guidelines. Brkige 6 then performs audto. video 
and data bridging in accordance witti predefined 
schemes that are agreed upon at the time of call setup. 
The bridged audio, video and data bit streams again are 
sent to ttie destination end stations via ISDN routers. 
The ISDN routers perform IP muttidEistirq in sending ttie 
multimedia traffic if necessary. As an alternative to IP 
multicasting, ttie ITU Rec T.I 25 multipoint communica- 
tion service (MCS) also can be used. 

The protocol architecture for end-to-end communi- 
cations for multimedia conferencing involving computer 
1-1 and multimedia bridge 6 is explained in connection 
witti Figure 2. The protocol stack 13-1 running on com- 
puter 1-1 has ttie entities GCC. MCS, H.225.0/RTP, 
TCP, UDP. IP. Logical link control (LLC), and medium 
access control (MAC) between ttie media stream 
(audio, video, and/or data) and a physical layer Simi- 
larly, profocol stack 13-2 of multimedia bridge 6 has ttie 
entities GCC. MCS. H,225.0/RTP. TCP. UDP, IP, LLC, 
and MAC between the media stream (audio, video 
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and/or data) and a physical layer. ISDN routers oT^I and 
3-2 communicate between oonputer 1 -1 and bridge 6 
using internet protocol (IP) (14-1. 14-2) Via nonguaran- 
teed quality of sendee local area network 2-1 and net- 
work hub 5-1. Network 2-1 and hub 5-1 have protocol 
layers LLC and MAC (1 7-1 , 1 7-2). ISDN routers 3-1 and 
3-2 encapsulate the IP over the point-to-point protocol 
(PPP or the point-to-point nrwitipoint (MP)) to transfer 
audio, video an*or data over ISDN links 4*1 and 4-2 to 
communicate over the ISDN wide area network 5. 
Router 3-1 is connected to a circuit switching ISDN 
node of network 5. 

The detailed architecture of computers M through 
1-5 is shown in Figure 3. The communication between 
the upper layer appFications 100 (e.g.. aucfio. vkJeo, 
data, system control, and/or other services ) and GCC 
1 06. MCS 1 07. or transport interface 1 01 can take place 
directly as needed. However, GCC 106 communicates 
with the transport interface via MCS 107 only in accord- 
ance with the ITU Rec. T.I 20 standard. Transport inter- 
face 101 makes the upper layer entities transparent 
from the details of the lower layers. Transport interface 

101 can be WinSock Forum's WINSOCK 2. Multimedia 
Communication Forum's (MMCF) TSI. or other inter- 
faces. 

The upper layer applications 100 may contain audio 
100-1. video 100-2, data 100-3, and/or other services 
100-4. Audio 105 arxJ video 103 coming from upper 
layer applications 100 or lower layer entity H.225.0/RTP 
108 are synchronized Into an entity known as synchro- 
nization 104. Both intramedia and intermedia synchro- 
nization services are perfomied by entity 104. The 
reference master time dock to provide synchronization 
is received from ISDN WAN 5 based bridge 6. The LAN 
based audio and video maintains synchronization by 
receiving dock signaling from ttie guaranteed quality of 
servtee ISDN WAN based bridge 6. No performance 
degradation occurs for any media sti^eam when tt is 
transferred over ISDN WAN 5. A synchronization algo- 
rrthm in accordance with tills irivention (Figures 5-8) 
maintains synchronization for different media. Conrput- 
ers 1-1 ttvough 1-5 are free to use ttieir own synchroni- 
zation methods. However, bridge 6 uses the 
synchronization algorittim described in connection witii 
ngures5-8. 

Audio 105 and video 103 bit streams coming out of 
the upper layer applications are packetized in accord- 
ance with tiie ITU Rec. H.225.0 protocol in entity 108. 
H.225.0 is very similar to ttie lETF's RTP protocol. 
There can be almost direct mapping between H.225.0 
and RTR The implementation details of tNs mapping 
function are not a part of Hiis invention. The audio and 
video packets encapsulated in H.225.0 are transfeaed 
using lETFs UDP protocol in entity 109. Similariy. data 

102 and system control 106 traffic are encapsulated 
over ttie lETFs TCP protocol in entity 1 10. The system 
control traffic entity 106 indudes control, call control, 
and RAS (Registration, Administration, and Status) as 


specified in tiie ITU Rec. H.245 and H.225.0 protocols. 
However. RAS ti-affic is trarsferred using tiie UDP pro- 
tocol. Botti UDP and TCP packets again are transfered 
usng lEFTs IP protocol in entity 111. The IP packet is 

5 " tiiei? transferred over the LAN (e.g.. LAN 2-1 ) using ttie 
LAN^protocol LLC in entity 1 12 and LAN protocol MAC 
in entity 113. 

If ttie packets come from the LAN to ttie computer 
(e.g.. LAN 2-1 to computer 1 -1). a similar de-encapsuta- 

10 tion of packets takes place from MAC in entity^13 to 
LLC in entity 112. from LLC in entity 1 12 to IP in entity 
111. from I P in entity 1 1 1 to U D P in entity 1 09 or to TCP 
in entity 110, and from UDP in entity 109 to the H.225.0 
protocol in entity 108. Audio 105 and video 103 bit 

75 stireams are recovered from the H.225.0 protocol in 
entity 108. The Int^-media and intramedia synchroniza- 
tions are performed for ttie recovered bit streams in 
entity 104. The synchronization services in ttie uf^er 
layer (atxsve the transport Interface 101) can also be 

20 provided, but tills service is not part of this invention. 

The protocol architecture for bridge 6 is ducted In 
Rgure 4. This architecture is similar to what has been 
shown in Figure 3. Bridge 6 has ttie~MCU and gate- 
keeper (GK) functionalities. The MCU functions of 

25 bridge 6 include bridging for audio, video and/or data 
similar to the functions stated for multipoint controller 
(MC) and multipoint processor (MP) of ttie ITU Rec. 
H.323 standard. The GK functions of bridge 6 include 
address translation, admission control, bandwkith con- 

30 trol. call control signaling, call auttiorization. barxiwidtti 
management, call management, directory services, and 
ottiers. However, bridge 6 works as ttie master for com- 
puters 1-1 tiirough 1-5 connected to the LANs in ttie 
customer premises (as shewn in Figure 1) from system 

35 controlling point of view. Bridge 6 is connected to ISDN 
router 3-2 via a dedicated LAN hub (as shown in Rgure 
1) and provides a guarantee for all performance param- 
eters. The packets coming to bridge 6 from the LANs 2- 
1 and 2-2 are de-encapsulated from MAC In entity 213 

40 to LLC in entity 212 to IP in entity 21 1 to UDP in entity 
210 or TCP In entity 209 as explained earlier. Botti Inti-a- 
media and intermedia synchronization for audio 207 
and video 205 sti-eams recovered from the H.225.0 
packets in entity 208 are performed In an entity known 

45 as synchronization 206. A synchronization algorittim 
described in Figures 5 through 8 inproves p^formance 
specifically targeted for the real-time traffic coming out 
of ttie LANs 2-1 and 2-2 and hub 5-2. Data, system con- 
trol, and ottier bit streams de-encapsulated from ttie 

so TCP packets in entity 209 are sent to the upper layer 
services. 

Audio 207 and video 205 bit streams then are trans- 
ferred to the higher layer services to perform media 
bridging for audio 200-1. video 200-2. and data 200-3 
55 bridging similar to specifications defined in ttie ITU Rec. 
H.323 standard. Every conferee sets up ttie communi- 
cation for multipoint multimedia conferendng via bridge 
6. A point-to-point communication flow is set up 
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between bridge 6 and each end station or end system 
partidpating in the conference (e.g.. computer 1-1 and 
1 -4). Audio bridging and video bridging are performed in 
entities 200-1 and 200-2 in accordance to the criteria 
agreed upon by the participating parties. For example. 5 
bridge 6 can provide either video switching or video mix- 
ing. Video switching is the process of selecting the 
video that bridge 6 outputs to compute's 1-1 through 1 - 
5 from one source to another. The criteria used to make 
the switch may be determined through detection of w 
change in speaker (sensed by the associated audio 
teveO or thoi^h control according to the H245 stand- 
ard. Video mixing is the process of formatting mae than 
one video source into a video stream that bridge 6 out- 
puts to computers 1-1 through 1-5. The details of the 75 
mixing criteria by the bridge 6 is not a part of this inven- 
tion. Brklge 6 can prepare N audio outputs from M audio 
irputs by switching, mixing or a comtxnation of these. 
Audio mixing requires decoding the input audio to linear 
signals (pulse coded modulation or analog), perfbmiing 20 
a linear comUnation of the signals, and recording the 
result to the appropriate audio format The details of 
such audio mixing atso^are not a part of this invention. 
Bridge 6 processes data in accordance with the ITU 
Rec. T.I 20 standard, and has functions like GCC in 25 
entity 201 and MCS in entity 202 as shown in Rgure 4. 

Rgures 5 through 8 desaibe a preferred form of 
algorithm to provide intra- and inter-media synchrcMiiza- 
tion to improve performance for the real-time audio and 
video trafffc over the networks shown in Rgure 1. The 3o 
algorittim detects the absence of properties needed for 
nmiKimedia conferencing, such as whether appropriate 
audio and vkjeo packets have been received, whether 
such packets are synchronized, and whether a com- 
plete video frame is ready. The algorithm is executed by 35 
synchronization entity 104 of bridge 6 (Figure 3). The 
process begins at step 500. while the initialization is 
done in step 501 . The variables shown in step 501 rep- 
resent tiie following: 

40 

i, j, k = dummy variables representing an integer 
number 

M » Total number of audio packets in a given 

audio segment 
N = Total nunr*>er of video packets in a given 45 

video frame 

F = Total number of video frames per second 

B = Total number of bits per video frame 

S s Total nunr^er of audio sample per sec- 

ond 50 
Q = Total numt^er of bits per audio sample 

T a Playout time 

l-i= Reference dock time 

Aq5 (k) = Total numt>er of bits in audio segment k 
Vfb (k) = Total number of bits in vkJeo frame k 55 
♦T = A fixed increment in playout time 

Vth = A fixed tiireshokJ value for ttie inter-amval 

time of the video packet 


Ath = A fixed threshold value for the into'-arrival 

time of the audio packet 
Ap(i) = Audio packet of i-th time interval 
Vp(i) = Video packet of i-th time interval 
Dj = Inter-arrival time of tfie j-th vMeo packet 

Cj = Inter-arrival time of the i-th audio packet 

Va = Average inter-arrival time for ttie video 

packet 

Aa = Average inter-arrival time for ttie audio 

packet 

Si = Initial system time 

Sf = Rnal systOT time 

P= New playout time 


The measurement of inter-arrival time for audk> and 
video packets tor a given playout time starts at the same 
time in step 501. The average inter-arrival times for 
video and audio packets for each source are estimated 
as shown in steps 503 and 502, respectively: 

N 

Va=(1/N)X;Dj 

jol 

M 

Aa = (1/M) 

I0I 


In step 504. the average irrter-anrival time of tiie 
video packet for given playout time is conpared to tfiat 
of the thtreshold value of ttie video packet inter-anival 
time. If Vq is greater than V^. bridge 6 notifies ttie 
source as indicated in st^ 506. Obtaining tiie notifica- 
tion from bridge 6. the source checks whettier ttie 
shared LANs, switches, ISDN routers, or ttie equipment 
are degrading tiie performance The source tries to rec- 
tify the problems or accepts ttie degraded mode of oper- 
ation. In step 505, audio packet inter-arrival time also is 
compared and actions similar to those descrtoed for 
st^s 504 and 506 also are taken as stated in step 507 
if Aa is greater ttian Ath. 

In step 508. the new playout time P is estimated by 
comparing ttie values of T and Vg. The maximun. of tfie 
two values is chosen as new playout time R In step 509, 
system initial time Sj is set to reference clock time 1^ ttiat 
is being maintained by ISDN WAN 5. System final tome 
Sf is set by inaementing the system initial time by ttie 
playout interval: S| = S, + P . 

In step 510. the total bits for audio segment A^{k) 
and video frame Vfb(k) wittiin ttie given playout time are 
estimated: Asj,(k) = SGP and Vft,(k)aFB. It. is 
expected that ttie playout time will be very ck)se to one 
video frame time. Audio and video are th^ processed 
simultaneously through a series of steps descvbed in 
Rgures 6 and 7 as indicated in steps 511 and 512, 
respectively. 

In step 601 (Rgure 6). ttie algoritfim examines 
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whether all audio packets. (Ap(i). i = 1. M] are received 
within system final time, Sf. H all packets are not 
received within the scheduled playout time, dummy 
packets are created to replace the unreceived packets 
(step 602). The creation of the dummy packets mini- 
mizes errors as far as practicable. Audio segment 
As5(k) then is formed (step 607). 

In step 603, the algorithm examines whether any 
audio packets received in step 601 do not t«Iong to 
audio segment The packets that do not belong 
to this audio segment are pre-buffered (step 605) if the 
packets could be used in the next segment. In step 607, 
audio segment is created only with those packets 
that belong to that segment In step 610. the audio proc- 
ess, after aeation of the segment, follows a series of 
steps as indicated in Figure 8. 

Similarly, in step 701 of Figure 7, the algorithm 
examines whether ail video packets. (Vp 0), j = 1. N] 
are received within system final time. S|. If all packets 
are not received within the scheduled playout time, sys- 
tem final time Sf is incremented by a small fraction of 
playout time (step 702) ♦! to set the new system final 
time: 

On the other hand, if all video packets are received 
wittiin system final time S|^. the algorithm examines 
(step 703) whether any video packets not belonging to 
video frame V(b(k) are received. If no excess packets 
are received, the vkJeo process is sent to st^ 805 of 
Figure 8 to follow a series of steps as indicated in step 
712. However, if there are excess video packets in video 
frame Vft,(k), all excess video packets are dropped (step 
706). and the process is sent to step 802 of Figure 8 as 
indicated in step 709. 

In st^ 704, ttie algorithm again examines whetfier 
all vkJeo packets. (Vp 0). j = L N] aie received within 
new system final time. Sf. If there are excess video 
packets in, vkJeo frame, Vto(k) the excess video packets 
are dropped (step 705) and the process is then sent to 
step 802 of Figure 8 as shown in step 709. If all vkJeo 
packets are not received within the new syst©n fiiral 
time, the process is sent to step 707. 

In step 707. the algorithm exanrtines whether 
dummy video packets can be created to fill up tfie gaps 
of video frame, Vft,{l<>, with minimum errors. If it is possi- 
ble to create video frame Vfo(k) using dummy packets, 
the process is then s&M to step 801 of Rgure 8 (step 
713). However, all video packets [Vp 0*).i = 1. NJ of 
video frame Vf5(k) are dropped (step 708). if this frame 
cannot be created due to non-availability of the video 
packets, and the process is sent to step 802 of Rgure 8 
as shown in step 709. 

In step 800 of Rgure 8, the audio process (from 
step 610 of Rgure 6) joins to the video process in step 
806. 812 or 810 depending on the video events for inter- 


media synchronization between the audio segment and 
video frame for playout storage, or processing as 
needed by the multimedia applications. 

In step 810, a video frame (from step 712 of Rgure 

5 7) and an audio segment (from step 61 0 or Figure 6) are 
synchronized for playout storage, or processing. In step 
814, the algorithm examines whether the complete 
video frame containing the necessary video packets 
along with the synchronized audio segment is ready. If 

w ' tfi^M^^Sme is re^y, tfie process is completed (step 
815); othenwise the process is repeated by going back 
to step 514 of Rgure 5 as indicated in step 81 1 . 

Step 802 shows tfiat the video process received 
from st^ 708 go^ to step 812, where audio segment, 

75 Asb(k) IS ac^usted to new system time Sf to which a new 
video playout time has been set In step 807, the audio 
segment is kept as a continuous reference medium for 
playait. storage, or processing although the video 
frame or packets have been dropped. 

20 In step 806. intermecfia synchronization between 
tiie audio (from step 61 0 of Figure 6) and video process 
(eittier from step 713 of Rgure 7 or from step 807) are 
performed for playout, storage or processing, and tiie 
processes are tiien sent to step 808. 

25 , In step 808, the synchronization to tfie new playout 
time is initialized, since tiie video frame playout time has 
been changed from its original value. In st^ 813, tiie 
algorithm examines whether the complete video frame 
containing the necessary video packets along with tiie 

30 synchronized audio segment is ready. If the video frame 
is ready, the process is completed (step 815); othenwise 
tf^e process goes to st^ 809 where botii audio and 
video processes then are reinitialized, going back to 
step 513 (Rgure 5). 

35 After step 815, tfie processed multimedia signals 
compising a audio segment and video frame are trans- 
mitted to ttie uppa* layer applications. In the case of 
multimedia bridge 6 (Rgure 1), the audio, video and 
data signals are bridged in entities 200-1, 200-2, and 

40 200-3, respectively. The bridged multimedia signals are 
then processed tiirough entities like 203, 210, 209. 21 1 , 
213, and 214. Then ttie processed signals are transmit- 
ted through LAN hub 5-1, router 3-2 and ISDN WAN 5 
as previously described. A receiving LAN, such as 2-^. 

45 processes the audio segment and video frame signals 
as previously desatoed in connection with Figure 3. If 
necessary, synchronization entity 104 may execute tiie 
algorittim shown in Rgures 5-8 in order to improve per- 
formance. 

50 In accordance with one aspect of my invention, dif- 
ferent categories of packet mode multimedia conferenc- 
ing services can be provided: premium service, semi- 
premium service, and non-premium service. 

Referring to Rgure 1. in premium service, all con- 

55 ference participating end stations within a customer 
premises are considered to be connected to tfie 
switched nonguaranteed quality of service LANs where 
only ttie switched LAN hubs (e.g., hub 5-2) are shared. 
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The switched LAN hubs are directly connected to an 
ISDN WAN router (e.g., ISDN rout^ 3-3). The traffic 
and the performance characteristics within the ISDN 
routers will satisfy the desired requirements specified in 
the premium service criteria. The ISDN WAN based 5 
service provider wilt guarantee to maintain the superior 
quality of service of the packet mxie multimedia confer- 
encing services, if the shared switched hubs and ISDN 
routers operate within the given performance guide- 
lines, w 

In semi-premium service mode, there can be a mix 
of switched and shared nonguaranteed quality of serv- 
ice UVNs where the partic^ting end stations are con- 
nected, tn this mode, there can be a conditional 
guarantee of quality of service by the ISDN WAN based is 
packet mode multimedia conferencing sen/ice provider. 

In non-premium mode, no guarantee of the quality 
of service is provided by the ISDN WAN service pro- 
vider. The service provider uses its best effort to provkJe 
the packet mode multimedia conferencing services. 20 

Those skilled in the art reoognize that the prefen'ed 
entxxJiments may be altered and amended without 
departing from the true spirit and scope as defined in 
the appended claims. 

25 

Claims 

1 . In a system for provding real-time nujitimedia con- 
ferencing services to a dispersed plurality of loca- 
tions interconnected by a wide area network from 30 
multimedia signals originating at said locations hav- 
ing properties adapted for said services, said loca- 
tions employing nonguaranteed quafity of service 
packet switched local area networks which degrade 

at least one of said prq3erties, said metfKXj com- 35 
prising in combination the steps of: 

receiving said multimedia signals from said 
local area networks at saJd wide area network; 
bridging said nrxiltimedia signals in saki wkie 40 
area network; and 

transmitting said bridged multimedia signals 
over said wide area network to said plurality of 
locations, whereby the bandwidth required to 
transmit said multimedia signals between said 45 
plurality of locations is reduced. 

2. A method, as claimed in daim 1. and further com- 
prising the steps of: 

50 

detecting the absence of at least said one prop- 
erty in said multimedia signals in said wide 
area network; 

processing said nruiltimedia signals to compen- 
sate for the absence of at least said one prop- 55 
erty; and 

transmitting said processed multimedia signals 
to said locations over said wide area networK 


whereby the quality of said services at said plu- 
rality of locations is improved. 

3. A method, as claimed in daim 2. wherein said step 
of receiving comprises the step of receiving said 
multimedia signals by said wide area network arid 
wherein said wide area network comprises an 
ISDN network. 

4. '- A-nrrethDd: as claimed in daim 3. wherein said step 

of transmitting conrprrses the step of time division 
multiplexing said processed nuiltimedia signals for 
transmission on said ISDN network. 

5. A method, as claimed in daim 2, wherein sakl step 
of detecting conrtprises the steps of: 

estimating a first time period by which a first 
group of said multimedia signals is expected to 
be received; 

determining whether any of said multimedia 
signals within saki first group is absent at the 
end of said first time period; 
increasing said first time period to a second 
time period in the event any of said multimedia 
signals is at)sent; and 

determining whetiier any of sakJ multimedia 
signals within said group is absent at tiie end of 
saki second time perkxi. 

6. A method, as claimed in daim 5. wherein said st^ 
of detecting further comprises the step of determin- 
ing whether any of sakJ multimedia signals received 
within said first time period belong to a second 
group different from said first group. 

7. A method, as claimed in daim 6. wherein said step 
of processing comprises the step of deleting at 
least some of said multimedia signals belonging to 
said second group. 

8. A method, as claimed in daim 6. wherein said step 
of processing comprises the step of storing at least 
some of saki multimedia signals belonging to said 
second group. 

9. A method, as claimed in daim 5. wherein said step 
of processing comprises the step of creating one or 
m>re dummy groups of multimedia signals for saki 
first group in the event that multimedia signals 
within said first group are absent 

10. A method, as claimed in daim 5. wherein said step 
of processing furth^ comprises the step of syn- 
chronizing said multimedia signals. 

11. A method, as claimed in daim 5. wherein saW mul- 
timedia signals comprise video signals and audio 
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signals and wherein said step of processing com- 
prises the step of synchronizing said video signals 
and said audio signals. 

1 2. A method, as claimed in daim 5. wherein said mul- 
timedia signals canprise video signals and audio 
signals and wherein said step of detecting com- 
prises the steps of: 

estimating a first time period by which a first 
segment of said audio signals is expected to be 
received; 

determining whether any of said audio signals 
within said first segment is absent at the end of 
said first time period; , 

increasing said fir^ time period to a second 
time period in the event any of said audio sig- 
nals is at>sent; 

detemnining whether any of said audio signals 
within said segment is absent at the end of said 
second time period; 

estimating a third time period by which a first 
frame of said video signals is expected to be 
received; 

detenmining whether any of said video signals 
within said first frame is absent at the end of 
said third time period; 

increasing said third time period to a fourth 
time period in the event any of said video sig- 
nals is absent; and 

determining whether any of said video signals 
within said frame is absent at the end of said 
fourth time period. 

1 3. A method, as claimed in claim 1 2, wherein said step 
of detecting further comprises the steps of: 

determining whether any of said audio signals 
received within said first time period belong to a 
second segment different from said first seg- 
ment; 

determining whether any of said video signals 
received within said third time period belong to 
a second frame different from said first frame, 

14. A method, as claimed in claim 13, wherein said step 
of processing comprises the steps of: 

storing any of said audio signals belonging to 
said second segment; and 
storing any of said video signals belonging to 
said second fraune. 

1 5. A method, as claimed in daim 13. wherein said ^ep 
of processing comprises the step of deleting at 
least some of said video signals belonging to said 
second frame. 


1 6- A method, as claimed in daim 1 3, wherein sakJ step 
of processing further comprises the step of syn- 
chronizing said audio signals within said first seg- 
ment with said video signals within said first frame, 

5 

17. A method, as claimed in claim 2, and further com- 
prising the steps of: 

rec&ving said processed multimedia signals at 
w one of said locations over said wide area net- 

detecting the absence of at least said one prop- 
erty in said muttimecfia signals at said one loca- 
tion; 

15 processing said one location multimedia sig- 

nals to compensate for the absence of at least 
said one property; and 

using said processed one location multimedia 
signals to aeate audio visual services at said 
20 one location. 

18. A method, as claimed in daim 17. wherein the step 
of detecting the absence of at least said one prop- 
erty in saki one locatipn multimedia signals com- 

25 prises the steps of: 

estimating a first time period by which a ftrst 
group of said one location multimedia signals is 
expected to be received; 

30 determining whether any of said one location 

multimedia signals within said first group is 
absent at the end of said first time period; 
inaeasing said first time p&nod to a second 
time period in the event any of said one location 

35 multimedia signals is absent; and 

determining whether any of said one location 
multimedia signals within said group is at»sent 
at the end of said second time period. 

40 1 9. A method, as claimed in daim 1 8. wherein said st^ 
of detecting the absence of at least said one prop- 
erty in Sciid received one location multimedia sig- 
nals further comprises the st^ of determining 
whether any of said one location nuiltimedia signals 

45 received within said first time period belong to a 
second group different from said first group. 

20. A method, as daimed in claim 1 9, wherein said step 
of processing said one location multimedia signals 

50 comprises the step of deleting at least some of said 
one location nruiltimedia signals thonging to said, 
second group. 

21 . A method, as daimed in claim 1 9. wheran said step 
55 of processing said one location multimedia signals 

comprises the step of storing at least some of said 
one location multimedia signals t>elonging to said 
secortd group. 
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22. A methcxJ. as daimed in daim 1 8, wherein said st^ 
of processing said one location multimecfia signals 
comprises the step of creating one or more dummy 
groups of multimedia signals for said first group of 
one location multimedia signals in the event that 5 
multimedia signals within said first group of one 
location multimedia signals is at)sent. 

23. A method, as daimed in daim 1 8. wherein said step 

of processing sakJ one location multimedia signals w 
further comprises the step of synchroiizing said 
one location multimeda signals. 

24. A method, as daimed in claim 1 . wherein said loca- 
tions employ switehed local area network hutjs and is 
further comprisir^g the steps of: 

sharing only said switched local area network 
hubs; and 

operating said switched local area network 20 
hut)S within predetermined performance guide- 
lines, whereby superior multimedia conferenc- 
ing services are provided with guaranteed 
quality of service parameters. 

25 

25. A method, as daimed in claim 1 . wherein said loca- 
tions employ a mix of shared and switched local 
area networks, whereby improved multimedia con- 
ferendng services are provided with conditional 
guarantee of qiality of service parameters. 30 
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